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Description 

Psychometric Creditworthiness Scoring 

for Business Loans 

Background of Invention 
[000 1 ] 1. Field of the Invention 

[0002] The invention relates generally to the evaluation of credit- 
worthiness of organizations, including, for example, busi- 
ness borrowers. In particular, the invention relates to an 
automated business creditworthiness scoring system and 
method that uses predictive modeling to perform pattern 
recognition and classification in order to assess the im- 
pact of the personalities of individuals associated with the 
business on the business" creditworthiness. 

[0003] 2. Description of the Related Arts 

[0004] | n the following discussion, the terms "business," 

"loan, ""creditworthiness, "and "managers"will be used for 
illustrative purposes; however, the techniques and princi- 
ples discussed herein apply to other types of organiza- 



tions and prospective business relationships and transac- 
tions that require risk management and an evaluation of 
the prospective counter-party, such as equity invest- 
ments, charitable donations, business partnership, vendor 
relationship, franchise relationship, channel distribution 
and production agreement. 

[0005] p r j or to the widespread use of consumer credit scores 

starting in the 1980's, lenders were loath to make unse- 
cured loans to even middle class consumers. Today, 
thanks to the availability of comprehensive credit bureaus 
and effective credit scores, US credit card lines of credit 
alone total nearly $4 trillion. An entire industry was en- 
abled by the development of quantitative credit risk man- 
agement technology. 

[0006] whereas managing risk and origination decisions for un- 
secured consumer loans is largely a solved problem there 
is currently no analogous tool for business loans. Business 
rating organizations such as D&B provide some informa- 
tion to lenders, but have been unable to duplicate the ef- 
fectiveness of consumer credit reports and scores. Unlike 
consumer loans, past credit behavior of companies is not 
as strong a predictor of future creditworthiness and re- 
porting is very sparse. Consequently, the usefulness of 



business credit reports is often limited to the occasional 
derogatory item that may be identified. 
[0007] Lenders, and others who invest in businesses, often ob- 
serve that the most important factor in their lending deci- 
sion is the way they feel about the management of the 
borrowing company. Are the borrowers too aggressive 
when risking other's money? Are they too conservative to 
make a business successful? Are they ethical? Do they 
possess fortitude of character? Are they reliable? When 
facing future difficulties, will they cash out as much as 
possible and leave the lenders exposed, or will they fight 
and make sacrifices to keep their obligations? In contrast 
with the impersonal information described earlier, these 
personal characteristics are not currently metered or ana- 
lyzed quantitatively. Lenders employ credit officers whose 
task is to evaluate the business' prospect using imper- 
sonal data, to perform due diligence evaluation of the 
supplied information, and, ultimately, to develop an opin- 
ion with regard to the personal characteristics of the man- 
agement team. 

[0008] Quantitative metering of personal characteristics, while 

unfamiliar to the financial industry, is a well-explored do- 
main in psychology. Psychometrics, as this domain is 



known, has been researched and analyzed statistically and 
qualitatively for nearly a century. Psychometric tests are 
extensively used across a wide spectrum of applications 
including, among many: clinical psychology, personality 
typing, pre-employment screening, management of mili- 
tary manpower, marketing and segmentation, marriage 
counseling, education, and career planning. Personality 
typing tests, unlike exams of skills or domain knowledge, 
identify characteristics such as introversion-extroversion 
tendencies by asking questions such as "Do you prefer to 
be in a crowd or with only a few friends?" The Myers- 
Briggs personally typing test, for example, is continuing 
to grow in popularity nearly 70 years after it was intro- 
duced. 

[0009] Creditworthiness of Business Loans 

[0010] p r j or to the widespread use of consumer credit scores 

starting in the 1980's (credit scores existed as early as the 
late 1950's, but they were not yet used extensively); 
lenders were loath to make unsecured loans to even mid- 
dle class consumers. Unless one had mortgaged assets 
(such as a home or a car) or government guarantee (such 
as for a student loan), lenders expected the borrower to 
demonstrate a very strong financial position. It was some- 



times quipped that the only people able to obtain unse- 
cured loans were those who could prove they do not need 
the money. 

[001 1] The problem of how to manage and make origination de- 
cisions for unsecured consumer loans has largely been 
solved with the combination of effective credit bureaus 
and credit scores such as those provided by Fair Isaac and 
Company (commonly referred to as "FICO Scores"). Infor- 
mation that was available on consumers' credit reports at 
the time of loans origination was correlated with the sub- 
sequent performance of the respective loans to identify 
predictive patterns. A statistically derived formula ex- 
pressing this correlation — in effect attempting to predict 
whether a loan will perform based on credit report infor- 
mation available at origination time — is at the core of 
every credit score. Lenders, eager to have access to credit 
reports and scores, have been willing to accept the costs 
and inconvenience of complete and timely reporting of 
consumer credit performance to the credit bureaus. The 
high degree of credit reports comprehensiveness farther 
enhanced the predictive abilities of the modeled credit 
scores. 

[0012] with the mass popularization of credit scores, effective 



risk management of unsecured consumer loans became 
possible. Lenders are able to extend loan offers to the 
public with terms that are specifically tuned to the credit- 
worthiness of each borrower and to know accurately and 
in advance the subsequent default rate. No lender, today, 
need ever again face a portfolio of unsecured consumer 
debt with unanticipated credit loss levels. The arena of 
competition between consumer lenders has thus shifted 
from being entirely that of fathoming credit risks to mar- 
keting techniques, attractive product offering, and en- 
couraging usage. Today, thanks to the availability of ef- 
fective credit scores, unsecured consumer loans — in- 
cluding credit card lines of credit — are ubiquitous in the 
middle class and even lower-income population seg- 
ments. Today, utilized US credit card loans alone total 
over $600 billion and committed lines of credit approach 

rd 

$4 trillion (source: FDIC, 3 quarter, 2002). A sizable in- 
dustry was enabled by the development of effective credit 
scores. 

[0013] whereas managing risk and origination decisions for un- 
secured consumer loans may be a largely a solved prob- 
lem, there is currently no analogous tool for business 
loans. For very small businesses — perhaps employing 5 



or fewer employees — a consumer credit score with re- 
spect to the principals may provide creditworthiness pre- 
dictivity. In such small businesses, the credit behavior of 
the company may be strongly related to that of the princi- 
pals. However, for larger businesses, the principals can 
assure that their personal credit is pristine — whether or 
not the business is doing well or managed prudently. 
[0014] Business rating organizations and those who accumulate 
information about the creditworthiness of businesses 
(such as Standard and Poor, Moody, and D&B — we will 
refer to these entities here as "Business Bureaus," some- 
what analogous to the credit bureaus that monitor con- 
sumer's credit behavior information) have developed a 
niche for providing some information to lenders, but have 
been unable to duplicate the effectiveness of consumer 
credit scores. Unlike consumer loans, good past credit be- 
havior of companies is not a strong predictor of future 
creditworthiness since companies often keep all their 
obligations until they are already in serious financial diffi- 
culties. Because predictivity is poor, the incentive for 
strong and timely reporting by lenders and other vendors 
is weak. In fact, it is counter-indicated: An entity that is 
owed money to by a financially tenuous business would 



not normally want to harm that business' access to credit 
elsewhere. With reporting sparse, at best, business bu- 
reaus are unable to provide effective scores even if pre- 
dictivity was more inherent. In fact, the only reason many 
lenders procure business bureaus report is to check for 
the off-chance that negative information is actually on the 
report. 

[0015] Current business loan scoring and origination decisions 
typically utilize impersonal information. Loan applicants 
are asked to supply financial reports, tax filings, and 
business plans. Information from business bureaus is 
added (particularly if derogatory information is pre- 
sented), and a score of the combined data is computed 
using the predictive modeling paradigm. Unfortunately, 
such scores are not considered very effective. Financial in- 
formation supplied by the applicant is suspect (it is typi- 
cally not audited, unless the borrower is publicly traded or 
very large; and even audited financial statements are 
sometimes misleading), business plans can be unrealistic, 
and business bureaus data is not very predictive. 

[0016] The state of the business lending market today is thus still 
in a pre-tool stage of development. Businesses — partic- 
ularly small or young enterprises with unproven cash flow 



and assets — find that they need to demonstrate a strong 
financial position to be considered for a loan. One might 
paraphrase the quip above to say that the only businesses 
that can get loans today are those that can demonstrate 
that they do not need the money. From a macro- 
economical point of view, this is a pity since businesses 
are ultimately productive enterprises with the potential to 
simultaneously grow capital and profitably service their 
loans. 

[0017] Psychometric Scoring and Personality Typing 

[0018] Sigmund Freud, credited with the development of psycho- 
analysis, was among the first medical and academic pro- 
fessionals to explore the complexity of human personal- 
ity. As early as the 1890's, he became convinced that 
one's personality is fixed long before the onset of adult- 
hood and remains unchanged thereafter. He also put forth 
a theoretical structure to explain how the personality is 
formed in the first place. Freud's two most important col- 
leagues, the Viennese physician Alferd Adler and the Swiss 
psychiatrist Carl Jung, broke with him in 1911 and 1912 
over disputes regarding the factors leading to the fixation 
of personalities. Adler went on to develop the individual 
psychology system which defined such concepts as inferior- 



ity complex, spoiled child, sibling rivalry, and adult lifestyle — all 
terms he coined. His views that care in child upbringing 
are critical throughout one's life — a corollary of the fixed 
personality hypothesis — remains unquestioned to this 
day. Jung took a more quantitative approach devising one 
of the earliest, effective personality instruments: The word 
association test. While taking very distinctive views of the 
mechanisms by which personalities are molded prior to 
becoming fixed, Freud, Adler, and Jung all agreed that 
personality is formed by the age of six. This conviction 
remains, to this day, an unchallenged cornerstone of 
modern psychology. 
[0019] Building upon Jung's thinking, journalist Katharine Briggs 
and her daughter Isabel Myers, starting in 1923, devel- 
oped a personality type indicator and associated mea- 
surement instrument. Known as the Myers-Briggs Type Indi- 
cator (MBTI), Form A had been copyrighted in 1943 and 
was revised and modernized a dozen times since. In 1975, 
starting with form F, Consulting Psychology Press became 
the publisher. Because of its long history and practicality 
in mainstream psychology, the MBTI has generated thou- 
sands of papers and over 1300 dissertations. Many other 
typing instruments have also been developed. The Journal 



of Psychological Types has now published 49 volumes de- 
voted to typological investigations. 
[0020] The MBTI instrument, encompassing a 93 items exam (in 
the 1998 Form M), produces a score on one of each of 
four dimensions: Extroverted -Introverted, Sensing-lntu- 
itive, Thinking-Feeling, and Judging-Perceiving. Each per- 
sonality type exhibits a preference along each of the four 
dimensions. Jungians offer the analogy of handedness — 
a right-handed person is not one who never uses the left 
hand, but rather one who prefers to use the right hand: 
Strongly or barely at all. A total of 16 potential personality 
type combinations are thus discernible with the MBTI test. 
The first dimension — a scale preference from extro- 
verted to introverted — is perhaps the best known and 
most pronounced personality type. Within the business 
world, extroversion has long been linked to effective sales 
ability; introverts have often succeeded in more solitary 
tasks such as accounting or computer programming. 
Likewise, the other dimensions of the MBTI have been 
linked to various aspects of job performance and success. 
Extensive quantitative validation research — for instance, 
finding a correlation between salesperson's performance 
and MBTI scores — has fortified the applicability of this 



instrument in the workplace environment. Today, more 
than 2.5 million MBTI tests, and countless other instru- 
ments, are administered annually in the workplace. 

[0021] (Historical recounting in this section is largely based on 
Psychological Testing at Work by Edward Hoffman) 

[0022] statistical Modeling and Credit Scoring Methodology 

[0023] Decisioning is the science of mass producing predictive ac- 
tions. Specified data that may have predictive value (predic- 
tive data) is collected prior to the decision point, a statistical 
model is utilized to generate a prediction with regard to 
some future event, and an action is taken based on the pre- 
diction. For example, in the consumer credit scoring do- 
main, a branch of decisioning, the predictive data includes 
information found in a prospective borrower credit report 
and application form, the future event is the eventual per- 
formance of the loan if originated, and the predictive ac- 
tion may be the decision of whether to originate the loan 
at all. In the invented psychometric credit scoring 
methodology, the predictive data would also include in- 
formation generated by psychometric instruments or in- 
terviews. We use the term "psychometric interview," herein 
to highlight the interrogative nature of the psychometric 
instruments with no intended loss of generality. 



[0024] The statistical modeling paradigm relies on the assump- 
tion that historical data present clues with respect to the 
manner through which predictivity may be obtained. Ar- 
guably, the process of human learning and human experi- 
ence gathering rely on the same assumption. The statisti- 
cal formalism divides the historical data into a collection 
of data point samplers — individual instances that are 
largely independent of each other and that would have re- 
quired individual predictions. In the credit scoring do- 
main, each loan, or sometimes each borrowing entity, may 
be thought of as a sampler. Each sampler is farther di- 
vided into component of information that would have 
been available prior to the decision point (the predictive 
data) and subsequent information that identify the out- 
come of the future event. Data can be considered historical 
only once enough time has passed that the outcome of 
the future event can be ascertained. 

[0025] Predictive data is typically processed so as to provide a 

collection of numer\ca\predictive variables. For example, the 
ratio of a borrower's income to its debt service load may 
be computed to form one such variable. Likewise, data 
identifying the ultimate disposition of the future event is 
processed into a numerical (sometimes binary) targetvari- 



able. An example of a binary target variable may be the 
determination of whether a loan proved to have been 
good (e.g., GOOD LOAN =1, BAD LOAN =0) where a loan 
is defined as good if it remains current or paid off within a 
specified time period, and bad otherwise. (An example of 
a non-binary — continuous — target variable may be the 
net present value of all profits and losses a loan generated 
over its lifetime; in the credit scoring arena, however, 
preference is often placed on direct GOOD/BAD binary 
target variables since measures such as profitability are 
influenced by many non-credit-related factors) 
[0026] a modeling data set is a collection of samplers where for 
each one there is a series of predictive variables and one 
target variable. A predictive model is a mathematical formula 
that provides a relationship between the target variable 
and associated predictive variables. Linear Regression, per- 
haps the best-known predictive model (although certainly 
not the best-performing one), for instance, is predicated 
on formulas where each predictive variable has a set 
weight in predicting the target variable and that the re- 
spective weights can be surmised from the historical data 
through algebraic techniques. More sophisticated model- 
ing techniques that are popular today include Logistic Re- 



gression (a modification of the Linear Regression method- 
ology that is specifically adopted to instances of a binary 
target variable), Neural Network (a modeling technique that 
is loosely modeled after the manner biological neurologi- 
cal system may operate), Binary Trees, Clustering Algorithms, 
and many others. Since the practical application of all 
these algorithms require the use of high-speed comput- 
ers, they are often referred to, collectively, as machine 
learning techniques. 
[0027] Historical data that was utilized in deriving the predictive 
model formula is often referred to as training data and the 
model derivation process itself is sometimes referred to 
as model training. As with human training and experience 
gathering, the model training process identifies instances 
of the information available prior to the decision point as 
contrasted with the ultimate outcome after the respective 
decisions. The validation of the performance of a predic- 
tive model is achieved by testing the performance of the 
model with respect to historical data that was not used in 
the training process. Such historical data, identified as test 
or validation data, represent a plausible simulation of the 
environment that will exist when the model will be de- 
ployed in a real system. For each sampler in the test data, 



the model formula is applied to the predictive variables to 
produce a prediction of the target variable. Since the test 
data is, itself, historical, the true target variable is known. 
The difference between the prediction and true target, the 
prediction error, is a measure of the performance of the 
model. A metering of the magnitude of the prediction er- 
rors performed on a historical test data set that is large 
enough to be statistically significant can provide a very 
accurate indication of the performance of the model. 
[0028] Creditworthiness models are often configured such that 

their predictions are presented in the form of a credit score. 
For instance, a score of zero may be defined as an indica- 
tion that a loan is unlikely to perform satisfactorily (a 
"BAD LOAN" outcome) and a score of 1000 may be de- 
fined as an indication that it is very likely to perform sat- 
isfactorily ("GOOD LOAN"). (FICO scores, as another ex- 
ample, use a 200-950 range). The performance of the 
model is then metered by how high it scores historical test 
data loans that subsequently proved to be good and how 
low it scores those that proved to be bad. A well-behaving 
model will exhibit a behavior where the proportion of bad 
loans among the population in each score band (score 
range) is decreasing as the score is increasing (e.g., there 



is a smaller percentage of BAD LOANS among those sam- 
plers whose score was in the 800-850 score band than 
the 750-800 band). 
[0029] The modeling paradigm presumption provides that a pre- 
dictive model that was trained using historical data and 
validated on separate, blind, historical data would — in 
the absence of extraordinary outside influences and mass 
behavioral changes — provide a comparable level of pre- 
dictivity with respect to subsequent live (operational) data. 
In a live score-oriented credit origination system, data 
that is available prior to the origination decision are pro- 
cessed to generate predictive variables, and the predictive 
model formula is then used to generate a credit score. 
The score is then directly interpreted as the likelihood of 
the originated loan ultimately becoming bad. If, given the 
terms of the loan, that likelihood is unacceptable from the 
lender's business point of view then the loan, as applied, 

is declined. 
Summary of Invention 

[0030] | n accordance with the present invention, there is pro- 
vided an automated system and method for scoring bor- 
rowing organizations creditworthiness, which uses a pre- 
dictive model to evaluate responses to an automatically 



administered psychometric interview by individuals asso- 
ciated with the borrowing organization and estimates the 
likelihood of repayment based on learned relationships 
among known variables. These relationships enable the 
system to estimate a probability of default for each 
prospective loan or other transaction presented in the 
form of a creditworthiness score. This score may then be 
provided as output to a human decision-maker involved 
in processing the transaction, or as part of a larger auto- 
mated loan decisioning system. The system periodically 
monitors its performance, and regularly redevelops the 
model utilizing subsequent loan performance data. 
[0031] The primary users of such a system, it is envisioned, 

would be banks and other financial institutions who lend 
to businesses. However, other entities facing relationship 
decisions whose ultimate outcome critically depends on 
the character of the management of the enterprise may 
also benefit from the systems. Such entities may include 
venture capitalists, shareholders, customers, vendors and 

others contemplating long-term relationships. 
Brief Description of Drawings 

[0032] Figure 1 is a block diagram of the process flow of an op- 
erational system 



[0033] Figure 2 is a block diagram of the process flow of a pilot 
system 

[0034] Figure 3 is a block diagram of the process flow of score 
fusion of psychometric and other credit scores 

[0035] Figure 4 is a block diagram of the process flow of a typo- 
graphical bureau 
Detailed Description 

[0036] The Figures depict preferred embodiments of the present 
invention for purposes of illustration only. One skilled in 
the art will readily recognize from the following discussion 
that alternative embodiments of the structures and meth- 
ods illustrated herein may be employed without departing 
from the principles of the invention described herein. 

[0037] Psychometric Scoring System 

[0038] Referring now to FIG. 1, there is shown a block diagram of 
a typical implementation of a Psychometric Scoring Sys- 
tem 100 in accordance with the present invention. Credit 
application information is applied to Psychometric Scoring 
System 100 via an Input Process 101. A Psychometric In- 
terview Administration Process 102 is utilized to adminis- 
ter a psychometric interview to selected members of the 
management of the credit applicant. In the preferred em- 



bodiment, the interview is administered via a secure inter- 
net interface and consists of a series of personality typing 
questions. Methods for administering questions via an In- 
ternet interface are readily known to one skilled in the 
arts. In the preferred embodiment, the selected members 
of management include the Chief Executive Officer or 
equivalent, the Chief Financial Officer or equivalent, and 
the most important management individual not already 
selected. The answers provided by the interviewees to the 
psychometric interview administered by Psychometric In- 
terview Administration Process 102, combined with credit 
application information gathered by Input Process 101 are 
supplied as inputs to a Credit Model Processing Module 
103. In the preferred embodiment, Credit Model Process- 
ing Module 103 utilizes a computer system and software 
written in the PHP language which may be run on a variety 
of conventional hardware platforms. In accordance with 
the software program instructions and data model, Credit 
Model Processing Module 103 outputs a score via Output 
Process 104 indicative of the creditworthiness of the ap- 
plicant. The score outputted via Output Processl04 is 
then utilized by the user lender to make a credit decision 
using Credit Decision Process 105. In the event that a de- 



cision to extend credit is made and credit is extended, 
subsequent loan performance data is collected via Loan 
Performance Data Harvesting Process 106. In the pre- 
ferred embodiment, data harvesting is achieved through 
monthly reports of loan performance for a portfolio of 
loans with the data archived on a computer disk system 
and managed by a commercially available data base man- 
agement system such as Microsoft Access. Data collected 
via the Loan Performance Data Harvesting Process 106 
and completed interviews generated by the Psychometric 
Interview Administration Process 102 is matched by Data 
Matching Process 107 to form a Historical Data File 108 
where each record represents one loan and contains the 
completed psychometric interview as well as subsequent 
loan performance information. In the preferred embodi- 
ment, the Historical Data File 108 is maintained on a com- 
puter disk system and managed by a commercially avail- 
able data base management system such as Microsoft Ac- 
cess. At regular intervals, the Historical Data File 108 is 
utilized to refine and redevelop a predictive model using 
Machine Learning Module 109. Methods for machine 
learning statistical modeling are readily known to one 
skilled in the arts and include such techniques as linear 



and non-linear regression, neural networks, clustering al- 
gorithms, decision trees, logistics regression, genetic al- 
gorithms, and others. In the preferred embodiment, a lo- 
gistics regression method is utilized. The redeveloped 
model produced using Machine Learning Module 109 is 
then supplied to Credit Model Processing Module 103 for 
use in subsequent credit scoring transactions. 

[0039] one may consider an implementation of such a system in 
a manner analogous to that of current consumer credit 
scores. Lenders, prior to origination, administer a psycho- 
metric test to members of the management of each bor- 
rowing business. The test and the administration process 
are designed to frustrate potential attempts to prepare for 
or otherwise cheat the intent of the exam (for instance, by 
making each exam unique and administered at the 
lender's facility). The completed exams, analogous to a 
consumer's credit report, are then the basis from which a 
score is computed. The score indicates the likelihood of 
satisfactory loan performance. 

[0040] whereas a simplistic model may focus on only one indi- 
vidual, or take a single-person exam-evaluation mindset, 
a more sophisticated approach may involve the analysis of 
the management as an integral team. Under that ap- 



proach, the completed exams are identified based on the 
positions of the individuals within management and the 
statistical model is trained with the benefit of this infor- 
mation. An even more sophisticated loan origination deci- 
sion process may seek to combine creditworthiness esti- 
mation (scores) derived by different processes to form one 
combined score. Such a combined score may incorporate 
currently-existing financial statements and business bu- 
reaus score with the proposed psychometric exam score. 

[0041] The invention operational system employs the statistical 
credit scoring methodology described above where pre- 
dictive variables are derived from psychometric interviews 
applied to members of the management of a borrowing 
institution. To be effective in a realistic setting, the system 
must also attain a degree of robustness against environ- 
mental factors, manipulation attempts, and outright 
cheating — while retaining a reasonable level of conve- 
nience to the impacted constituencies. 

[0042] Once the system is operational, a psychometric interview 
administration protocol would be followed. The protocol 
would determine which members of the management of a 
business applying for a loan would need to be examined, 
a secure manner by which to assure the identity of those 



examined, and methods to prevent individuals from ma- 
nipulating their own score (through studying or "cheat- 
ing"). 

[0043] identity assurance would likely be done through methods 
of direct physical security — such as requiring the inter- 
view to be administered at the lender's (or affiliate) site, 
where an ID check would be practicable. Independent No- 
tary Public providers might also be recognized in assuring 
proper identification of examinees when conditions make 
this potential option more practical (e.g., allow an inter- 
viewee to be administered the interview in the presence of 
a Notary Public and certify proper identity under oath). If 
identity assurance is not considered a concern, then an 
online or a telephone interview administration service may 
provide a level of convenience to the interviewee. 

[0044] jo reduce the ability of interviewees to prepare for inter- 
views, algorithms via which each administered interview is 
unique may be employed. A large pool of questions may 
be initially created, and each administered interview will 
draw upon only a small fraction of those questions. No 
two interviews will thus have more than a small number of 
questions in common. The psychometric profession has 
developed a quiver of proven methods to identify likely ly- 



ing. Methods such as "trick questions" (questions that 
honest examinees almost always answer in a particular — 
often not very self-complimentary — manner, whereas 
dishonest examinees sometimes answer differently) have 
a surprisingly strong track record in identifying dishonest 
interviewees. Less definitive methods — such as recog- 
nizing an unlikely preponderance of unusual correlation 
between answers to questions — may segregate those 
who answer untruthfully together with those whose per- 
sonality is extraordinarily atypical. If the predictive model 
determine that this "extraordinarily atypical" category of 
borrower is particularly risky, then it does not really mat- 
ter if some members of that category came to be such 
through dishonesty. 
[0045] whereas a simplistic model may focus on only one indi- 
vidual, or take a single-person exam-evaluation mindset, 
a more sophisticated approach may involve the analysis of 
the management as an integral team. Under that ap- 
proach, the completed exams are identified based on the 
positions of the individuals within management and the 
statistical model is trained with the benefit of this infor- 
mation. It may be that the best-performing management 
teams have specific combinations of management person- 



alities (for instance, a moderately aggressive Chief Execu- 
tive Officer (CEO) and a conservative Chief Financial Offi- 
cer (CFO)) and that the worst performing teams have other 
combinations (for instance, a team that is entirely com- 
prised of aggressive personalities). Fortunately, the inven- 
tion does not require us to qualitatively research and un- 
derstand these dynamics. If such correlation, in fact, ex- 
ists; then any reasonable modeling technique would have 
the effect of discovering and accounting for it. 
[0046] Once the proper psychometric interviews have been ad- 
ministered to members of the management of the bor- 
rowing business, the results of the measures will be used 
to generate predictive variables and those in turn, via the 
credit model, would be used to generate a credit score. 
The credit score would then be utilized within a lending 
policy instituted by the lender and constituted to achieve 
specified business objectives. For instance, a simplistic 
lending policy might simply decline all loans where the 
credit score is below a specified threshold. A more nu- 
anced policy might set multiple thresholds indicating risk 
level where loans must have more rigorous terms, require 
approval by a more senior loan officer, or demand stricter 
covenants. 



[0047] whatever the final credit decision may be, a record of the 
administered interview, together with the credit decision 
will be retained. That record will, subsequently, be ap- 
pended with data indicating the credit performance of the 
corresponding loan. Ultimately, once enough time has 
passed such that it may be determined, retrospectively, if 
the loan proved to have been good or bad, the record will 
become part of the historical data that will be used to 
train or test subsequent model versions. 

[0048] pj| 0 t Psychometric Scoring System 

[0049] Referring now to FIG. 2, there is shown a block diagram of 
a typical implementation of a Pilot Psychometric Scoring 
System 200 in accordance with the present invention. In 
the preferred embodiment, a Pilot Psychometric System 
200 is utilized initially to develop an initial credit model, 
with that credit model becoming the initial credit model of 
the Psychometric Scoring System 100 that is used by the 
Credit Model Processing Module 103. Credit application 
information is applied to Pilot Psychometric Scoring Sys- 
tem 200 via an Input Process 201. A Psychometric Inter- 
view Administration Process 202 is utilized to administer 
a psychometric interview to selected members of the 
management of the credit applicant. In the preferred em- 



bodiment, the interview is administered via a secure inter- 
net interface and consists of a series of personality typing 
questions. Methods for administering questions via an In- 
ternet interface are readily known to one skilled in the 
arts. The answers provided by the interviewees via the 
Psychometric Interview Administration Process 202 are 
combined with credit application information gathered by 
Input Process 201 and tagged as "Origination"by Data 
Preparation Process 203 with each loan represented by 
one tagged data file record. Contemporaneously, credit 
application information of loans that have subsequently 
failed to perform to the lender's satisfaction is also ap- 
plied to system 200 via an Input Process 204. A Psycho- 
metric Interview Administration Process 205 is utilized to 
administer a psychometric interview to selected members 
of the management of the creditor. In the preferred em- 
bodiment, the interview is administered via a secure inter- 
net interface and consists of a series of personality typing 
questions. Methods for administering questions via an In- 
ternet interface are readily known to one skilled in the 
arts. In the preferred embodiment, the psychometric in- 
terview administered by Psychometric Interview Adminis- 
tration process 205 is identical to the psychometric inter- 



view administered by Psychometric Interview Administra- 
tion process 202 and the selection methodology for inter- 
viewee members of management is also identical. In the 
preferred embodiment, the selected members of manage- 
ment include the Chief Executive Officer or equivalent, the 
Chief Financial Officer or equivalent, and the most impor- 
tant management individual not already selected. In the 
preferred embodiment, members of management of non- 
performing creditors are engaged in a renegotiation pro- 
cess, commonly known as a "Workout"with the lender and 
acceptance of the psychometric interview becomes part of 
the process. In the preferred embodiment, the psychome- 
tric interview is designed to select questions to which the 
answers are unlikely to change with the passage of time. 
The answers provided by the interviewees via the Psycho- 
metric Interview Administration Process 205 are combined 
with credit application information gathered by Input Pro- 
cess 204 and tagged as "Workout" by Data Preparation 
Process 206 with each loan represented by one tagged 
data file record. The data files prepared by Data Prepara- 
tion Processes 203 and 206 are combined to form Tagged 
Data File 207. In the preferred embodiment, the Tagged 
Data File 207 is stored on a computer disk system and 



managed by a commercially available data base manage- 
ment system such as Microsoft Access. Tagged Data File 
207 is used by Modeling Process 208 to train a statistical 
model that predicts the tag of each record utilizing the 
loan application and interview answers of the correspond- 
ing record. The predictions of the statistical model gener- 
ated by Modeling Process 208 are generated in the form 
of a credit score. Methods for statistical modeling are 
readily known to one skilled in the arts and include such 
techniques as linear and non-linear regression, neural 
networks, clustering algorithms, decision trees, logistics 
regression, genetic algorithms, and others. In the pre- 
ferred embodiment, a logistics regression method is uti- 
lized. All loans represented in the Tagged data Set 207 
are scored using the statistical model generated by Mod- 
eling Process 208 by Scoring Process 209 with the scores 
appended to each record to form Scored and Tagged Data 
File 210. In the preferred embodiment, the Scored and 
Tagged Data File 210 is stored on a computer disk system 
and managed by a commercially available data base man- 
agement system such as Microsoft Access. The Scored and 
Tagged Data File 210 is supplied to Origination Outlier 
Scrutinization Process 211 which is used to identify loans 



tagged as "OriginatiorTwhose score is more indicative of 
loans tagged "Workout" and scrutinizes the details of such 
loans for the cause of their non-conformity. In the pre- 
ferred embodiment, Origination Outlier Scrutinization 
Process 211 involves a computerized sorting and group- 
ing of the "Origination"-tagged record of the Scored and 
Tagged Data File 210 based on score, the selection of a 
score threshold, and a human analysis of all loans past 
the score threshold. Information gathered from Origina- 
tion Outlier Scrutinization Process 211 is utilized in the 
Final Approval Process 212, or subsequent tracking, of 
loans. Contemporaneously, the Scored and Tagged Data 
File 210 is supplied to Workout Outlier Scrutinization Pro- 
cess 213 which is used to identify loans tagged as "Work- 
out" whose score is more indicative of loans tagged "Origi- 
nation"and scrutinizes the details of such loans for the 
cause of their non-conformity. In the preferred embodi- 
ment, Workout Outlier Scrutinization Process 213 involves 
a computerized sorting and grouping of the "Work- 
out"-tagged record of the Scored and Tagged Data File 
210 based on score, the selection of a score threshold, 
and a human analysis of all loans not past the score 
threshold. Information gathered from Workout Outlier 



Scrutinization Process 213 is utilized in the Workout Pro- 
cess 214, or subsequent tracking, of loans. 
[0050] An operational scoring system 100, as described above, 
would score new loan applications using a model that was 
developed based on the experience of the performance of 
prior granted loans. The subsequently granted loans, in 
due course, will provide data for the training of future 
model revisions. While this overall operational process is 
self-sustaining, there may be difficulty in achieving the 
initial model. Since, at this time, lenders do not administer 
psychometric interviews to borrower, there is no historical 
data available from which to construct a credit model. If 
one were to begin administering psychometric interviews 
to borrowers today, one would still need to wait until 
enough time has passed so that those loans now originat- 
ing have enough track record to be classified as good or 
bad. This waiting period may take years. In the meantime, 
test administration would be strictly for research purposes 
with no origination decision impacted by these pre- 
model-development interviews. Arguably, the motivation 
of the borrowers and employees of the lending institution 
to follow precise protocols during this prolonged pre- 
benefit period may wear thin. Furthermore, until the first 



model is in production, there will be no way to optimize or 
tune the interviews. Poor interview questions (i.e., ques- 
tions with no incremental predictive value) and weak ad- 
ministration protocols will not be ferreted out until there 
is enough data to build the first model. If one imagines 
that a few iterations might prove necessary before an eco- 
nomically viable system is implemented, then this process 
might be long and awkward indeed. 
[0051] | n most predictive decisioning environment, this initial 
hurdle is unavoidable. The first model cannot be con- 
structed until enough samplers of pre-decision data have 
been collected and enough time to allow those samplers 
to become historical (by waiting for the predicted event to 
become manifest) has passed. Data collected after the 
predicted event is materially different — often through 
the impact of the event — from that which would have 
been available at decision time. For instance, a consumer 
credit model that attempts to predict personal bankrupt- 
cies using credit report information can not use the credit 
reports as they are when the bankruptcies are filed, but 
must rather use the credit reports as they are when the 
model would normally be used (i.e., at the time of loan 
origination). This is because the credit report at 



bankruptcy time would likely be materially different than 
at loan origination time and would likely contain many 
clues that would not have been available earlier. 
[0052] The only time this data collection waiting rule can be 

plausibly relaxed is when one can demonstrate that the 
collected data would not have likely changed materially 
during the waiting period. For example, if one were to 
build a predictive medical model were stable factors such 
as adult height and blood type are used to predict the 
likelihood of the onset of some medical condition (for in- 
stance, a heart attack), it is acceptable to measure those 
factors after the onset of the condition for the purpose of 
building a predictive model. Fortunately, in the instance of 
psychometric testing and personality typing, the weight of 
evidence indicates that adult personalities do not change 
even in the wake of dramatic or traumatic events (such as 
the loss of one's business might be). This stability of per- 
sonality opens the possibility of developing pilot models 
of economic value without the burdensome need for a 
waiting period. Such a pilot model could provide initial 
benefit while data is collected for the more rigorous sub- 
sequent revision (when truly historical data will be avail- 
able). 



[0053] one potential application of this thinking involves the ad- 
ministering, over a short period of time, of the psychome- 
tric interview to members of management of borrowers 
whose loans are in bad standing and also members of 
management of borrowers whose loans are in good 
standing. The former category might be found in a 
lender's loan workout section (a department dedicated to 
salvaging value from written-off, or otherwise poorly per- 
forming, loans). The latter category might be found 
among borrowers whose debt is in good standing for a 
sufficiently long time and who are in additional origination 
negotiation with the lender (for instance, for additional 
credit). It may even be possible to simply use any loan ap- 
plicant as a proxy for good-standing loans, since only a 
small percentage of granted loans ever become bad in any 
event. Both such groups are in a position of needing the 
lender's favorable actions and would thus likely comply 
with a psychometric metering mandate. 

[0054] Once such pilot data is collected, a model that attempt to 
predict, given the collected management psychometric 
data for each loan, whether it is from the pool of good or 
bad loans. Standard machine learning techniques, as de- 
scribed above, would be readily usable to construct such a 



model and the model could be immediately validated. As- 
suming the model exhibits a statistically significant pre- 
dictive capacity; those loans that the model mis-classifies 
would be candidates for actions. For instance, if the 
model, presented with a loan that came from the origina- 
tion ("good") pool, determines that it possess predictive 
characteristics more commonly found in the workout 
("bad") pool, this may be indicative of the need for farther 
credit evaluation actions. Likewise, if the model finds a 
loan from the workout pool to be more consistent with 
those found in the origination pool — that loan may be 
viewed as a more likely candidate for a successful work- 
out. By providing its prediction in the form of a score, the 
product will be readily usable in the formation of credit 
granting policy. 
[0055] Score Fusion System 

[0056] Referring now to FIG. 3, there is shown a block diagram of 
atypical implementation of a Score Fusion System 300 in 
accordance with the present invention. Credit application 
information is applied to Score Fusion System 300 via In- 
put Process 301. A Traditional Credit Scoring Technique 
Process 302 is used to generate a traditional credit score 
based on credit application information inputted via Input 



Process 301. Systems and methods for generating a tradi- 
tional credit scoring techniques are readily known to one 
skilled in the arts and are commercially available from a 
multitude of providers including Fair Isaac Corporation, 
D&B, and Moody"s. The Traditional Credit Scoring Tech- 
nique Process 302 may require additional information that 
can be obtained from Outside Data Sources 303. System 
and methods for accessing outside sources for credit re- 
lated information are readily known to one skilled in the 
arts and are commercially available from a multitude of 
outside data providers including Experian, Equifax, Tran- 
sunion, D&B, and Moody"s. Contemporaneously, credit 
application information inputted via Input Process 301 is 
applied to a Psychometric Interview Scoring System 304 to 
generate a psychometric interview credit score. In the pre- 
ferred embodiment, the Psychometric Interview Scoring 
System 304 are formed from the previously described 
Psychometric Scoring System 100 with the inputted credit 
application supplied via the previously-described credit 
application information Input Process 101 and outputted 
credit score as the previously-described score outputted 
via Output Process 104. The traditional credit score gen- 
erated by Traditional Credit Scoring Technique Process 



302 and the psychometric interview score generated by 
Psychometric Interview Scoring System 304 are combined 
by Score Fusion Process 305. System and methods for 
combining scores are readily known to one skilled in the 
arts and include such techniques as table look-up, linear 
and non-linear regression, neural networks, clustering al- 
gorithms, decision trees, logistics regression, genetic al- 
gorithms, and others. In the preferred embodiment, a ta- 
ble look-up with smoothing between grid points is uti- 
lized. The output of the Score Fusion Process 305 is a 
comprehensive credit score indicative of the likely credit- 
worthiness of the scored credit application and is out- 
putted by the system via Output Process 306. 

[0057] | t j S n ot the intent of this invention to completely displace 
all other manners of evaluating the creditworthiness of 
businesses. Even a management team with the most pris- 
tine personalities would have a hard time repaying loans if 
the revenue stream and business plan are inadequate. 
Current creditworthiness evaluation techniques — largely 
based on reviewing financial statements, business plans, 
tax filings, business bureaus data, and other accounting-ori- 
ented information — should not be discarded. 

[0058] when two statistical methodologies are based on radically 



different mechanisms, there is a lesser opportunity that 
their predictive power is duplicative. This property, re- 
ferred to as orthogonality, when present, allows for the 
combination of the two methodologies to gain a combined 
value. Credit scores formed through psychometric 
methodologies and through accounting-oriented ap- 
proaches have the opportunity to be orthogonal. 
[0059] Consider, for example, a situation where bad loans are 
rare (but individually expensive to the lender). Suppose 
some credit risk evaluation method is able to finger 30% 
of the originated loans as "risky" and, in fact, nearly every 
bad loan is destined to emerge from amongst the fingered 
loans. By itself, such a method may only be marginally 
useful — the lender is likely to find it too onerous to de- 
cline 30% of otherwise-approved loans to eliminate the 
credit loss problem. If a second credit risk evaluation 
method had exactly the same performance, it will be of 
equally marginal usefulness. If the two methods identify 
largely the same 30% of loans as "risky", then there is no 
value in combining the two methods. However, if the two 
methods are completely orthogonal, good loans that are 
(mis-)labeled by one method as risky are no more likely to 
be so labeled by the other method than other good loans. 



Consequently, only 9% (30% of 30%) of all loans would be 
labeled as risky by both methods. These 9% would con- 
tinue to include nearly every bad loan. The utility of the 
combined method, presumably, would be better than 
marginal. 

[0060] when two (or more) statistical predictive methods exhibit 
partial or complete orthogonality, it is possible to com- 
bine them into one method with a higher degree of pre- 
dictive power than either of the two methods alone. When 
the predictive methods produce scores, this combination 
process can be referred to as score fusion and the outcome 
can itself be in the form of score. 

[0061] Typographical Bureau 

[0062] Referring now to FIG. 4, there is shown a block diagram of 
atypical implementation of a Typographical Bureau 400 in 
accordance with the present invention. Requests for Psy- 
chometric Interviews are received via a computer network 
Request Receiving Process 401. In the preferred embodi- 
ment, requests for psychometric Interviews are made by 
the Psychometric Scoring System 100 when it becomes 
necessary to administer a psychometric interview by the 
Psychometric Interview Administration Process 102. Upon 
the receipt of a psychometric interview request by Request 



Receiving Process 401, Retrieval Process 402 queries Data 
Storage Facility 403 to retrieve, if available, a copy of a 
previously completed psychometric interview by the same 
individual interviewee. In the preferred embodiment, Data 
Storage Facility 403 is a high speed computer disk system 
managed by a commercially available data base manage- 
ment system such as the Microsoft Access system. A Veri- 
fication Process 405 establishes if the requirements to 
utilize the retrieved completed interview are present. In 
the preferred embodiment, the requirements include the 
completeness and existence of a completed interview, the 
absence of ambiguity regarding the exactness of the 
match of the identity of the interviewee, an limit on the 
time span since the retrieved interview was completed, an 
authorization provided by a competent party or parties at 
the time the retrieved interview was completed for subse- 
quent reutilizing, an authorization provided by a compe- 
tent party or parties at request time, and the appropriate- 
ness of the completed interview with respect to the re- 
quested interview. In the preferred embodiment, the time 
span requirement is no greater than seven years. In the 
preferred embodiment, an authorization by the intervie- 
wee given either at the time of the interview or at the time 



of the request for retrieval will satisfy the requirement for 
competent authorization. In the preferred embodiment, an 
identical social security number and a largely similar name 
accounting for marriage- and divorce-related name 
changes as well as non-unique spellings and shortening 
of some names is considered to satisfy the requirement 
for a non-ambiguous identity match. If the Verification 
Process 405 finds that all requirements are met, the sys- 
tem outputs the retrieved psychometric interview via Out- 
put Process 406. In the preferred embodiment, Output 
Process 406 delivers the completed interview through the 
same computer network and to the same computer pro- 
cess that initiated the initial request for psychometric in- 
terview through Request Receiving Process 401. If the 
Verification Process 405 finds that not all requirements 
are met, a psychometric interview is administered to the 
interviewee via Psychometric Interview Administration 
Process 407. In the preferred embodiment, the interview 
is administered via a secure internet interface and consists 
of a series of personality typing questions. Methods for 
administering questions via an Internet interface are read- 
ily known to one skilled in the arts. The system outputs 
the completed psychometric interview generated by the 



Psychometric Interview Administration Process 407 via an 
Output Process 408. In the preferred embodiment, the 
Output Process 408 delivers the completed interview 
through the same computer network and to the same 
computer process that initiated the initial request for psy- 
chometric interview through Request Receiving Process 
401. In addition, a copy of the completed interview gener- 
ated by the Psychometric Interview Administration Process 
407 is forwarded to the Data Storage Facility 403 to be 
available for future queries. 

[0063] The non-volatile nature of personality type would tend to 
suggest that repeated administration of a measurement 
interview to the same individual would produce similar re- 
sults and would thus be wasteful and unnecessarily incon- 
venient to all involved. A trusted repository of typographi- 
cal information would thus appear appropriate. 

[0064] one plausible implementation of a typographical bureau 

would combine the function of trusted repository with that 
of model building. Every time a member of the manage- 
ment of a loan-applicant is administered a typographical 
interview, all data generated by that event is forwarded to 
the bureau for safekeeping. In the event the same individ- 
ual is in a position where, again, a typographical exami- 



nation is required — perhaps as a consequence of another 
loan application — even if at the time the individual is 
representing a different borrower or applying to a differ- 
ent lender, the bureau could be requested to produce the 
result of the prior examination rather than engage in the 
hassle of a repeat exam. In addition, the lender would 
agree to provide the bureau with information regarding 
loan decisions and subsequent loan performance for 
those loans that were originated with the aid of the psy- 
chometric model. This data, in turn, will become the his- 
torical data from which future model revisions will be 
trained. 

[0065] Once such a trusted bureau is well established, it may 

serve other constituencies. At the request (or, at least with 
the consent) of the examinee, typographical information 
may be released to other entities that may have a legiti- 
mate interest: For example, employers, business partners, 
vendors, etc. 

[0066] 



